-
Notifications
You must be signed in to change notification settings - Fork 32
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Parse rustls.h to generate an API docs website #510
base: main
Are you sure you want to change the base?
Conversation
Not sure the best way to demonstrate this working without merging first. Probably running |
Odd. Out of time for today but I'll look into this soon. |
bde927d
to
f0c393e
Compare
Not so odd. I broke the |
Here's a preview I made with a test repo: https://cpu.github.io/rustls-ffi-docs/ There's definitely room for improvement. In particular there's no cross-linking yet, and I think a comprehensive edit pass through the existing comments in the .h would be helpful. |
f0c393e
to
a2b6c53
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Haven't reviewed the code in detail but the concept and the output look good to me 👍🏻
Thanks for taking a look, much appreciated 🙇 |
a2b6c53
to
1c2a3ec
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
sorry, these review comments were trapped in "finish your review" purgatory
On UNIX, use `tee` to get the output to stdout and `grep` the stream afterwards without using an output file. For Windows, use powershell's `tee` equiv (`Tee-Object`) and display the log file before grepping it. This will surface any errors that occur alongside the failed grep exit code.
A few elements of our public API had no associated docs. This is going to be flagged as an error by the API doc generation tooling, so let's fix it up-front with some short doc additions.
In a subsequent commit we're adding a docgen tool to the tools crate with a dep that requires a higher MSRV than the main crate. To make this less painful this commit excludes the tools crate from the workspace default list and skips usage of the ech-fetch tool for the MSRV runs.
This commit adds a `docgen` binary to the tools crate. This tool can generate a .json file describing the API based on parsing the `cbindgen` generated `rustls.h` header file using a tree-sitter C grammar. The produced JSON can in turn be used to generate markdown web documentation, or any other format required. Along with generating docs this tool applies some basic policy. In particular it requires that **all** public API items be documented or it will produce an error instead of the JSON data. We can extend this in the future to require (for example) describing arguments and return values in doc comments. The generated JSON has the following structure: ```json { "structs": [ .. ], "functions": : [ .. ], "callbacks": [ .. ], "enums": [ .. ], "externs": [ .. ], "aliases": [ .. ] } ``` Where each key is a general category of item: * Structure definitions * API function prototypes * Callback typedefs * Enum typedefs * Extern typedefs * Simple type alias typdefs Within each category items are described like: ```json { "anchor": "rustls-accepted", "comment": "A parsed ClientHello produced by a ...", "name": "rustls_accepted", "text": "```c\nstruct rustls_accepted\n```" }, ``` The anchor field is useful for creating anchor links that identify the item. The name is the actual name of the item. The comment is the C block comment content (with the block comment syntax removed, but other whitespace left as-is). Lastly the text field holds the actual text that defines the item in rustls.h (without comment) as a C formatted markdown code block.
1c2a3ec
to
d0eaa12
Compare
Filed #514 for this. There's something flaky going on for this test on Windows runners. |
This commit adds a Zola[0] website that can template the API docs JSON data into a nice website showing off our API. The choice of Zola (and the initial CSS/config) are inspired by the https://rustls.dev website which also uses Zola. To use: ``` cargo run --bin docgen > website/static/api.json cd website && zola serve open http://locahost:1111 ``` [0]: https://www.getzola.org/
Lifted from the equivalent config in the rustls repo, and customized for rustls-ffi.
d0eaa12
to
c183852
Compare
librustls: add missing API docs
A few elements of our public API had no associated docs. This is going to be flagged as an error by the API doc generation tooling, so let's fix it up-front with some short doc additions.
add docgen tools binary
This commit adds a
docgen
binary to the tools crate. This tool can generate a .json file describing the API based on parsing thecbindgen
generatedrustls.h
header file using a tree-sitter C grammar.The produced JSON can in turn be used to generate markdown web documentation, or any other format required.
Along with generating docs this tool applies some basic policy. In particular it requires that all public API items be documented or it will produce an error instead of the JSON data. We can extend this in the future to require (for example) describing arguments and return values in doc comments.
The generated JSON has the following structure:
Where each key is a general category of item:
Within each category items are described like:
The anchor field is useful for creating anchor links that identify the item. The name is the actual name of the item. The comment is the C block comment content (with the block comment syntax removed, but other whitespace left as-is). Lastly the text field holds the actual text that defines the item in rustls.h (without comment) as a C formatted markdown code block.
add an API docs website powered by Zola
This commit adds a Zola website that can template the API docs JSON
data into a nice website showing off our API.
The choice of Zola (and the initial CSS/config) are inspired by the https://rustls.dev/ website which also uses Zola. In the future we can add other pages (e.g. higher level tutorial/getting started content).
To use:
ci: add docs page deployment workflow
Lifted from the equivalent config in the rustls repo, and customized for rustls-ffi.
Updates #215